Iterative qualitative approach to establishing content validation of a patient-reported outcome measure for arm lymphedema: the LYMPH-Q Upper Extremity Module

Background Breast cancer-related lymphedema (BRCL) is one of the most common causes of upper extremity (UE) lymphedema in developed nations and substantially impacts health-related quality of life. To advance our understanding of the epidemiology and treatment of BRCL, rigorously developed and validated patient-reported outcome measures (PROMs) are needed. This study aimed to demonstrate the iterative content validity of a modular UE lymphedema-specific PROM called the LYMPH-Q UE module. Methods A multi-step iterative qualitative approach was used. Semi-structured interview data from in-depth qualitative interviews with adult women (18 years and older) with BCRL were used to develop the first set of the LYMPH-Q UE scales. The content validity of these scales was demonstrated with patient and clinician feedback. Over the course of cognitive debriefing interviews, additional concepts of lymphedema worry and impact on work were identified as missing from the LYMPH-Q UE module. Subsequently, two new qualitative studies (a focus group and in-depth concept elicitation interviews with patients) were conducted, and two new scales were developed to measure lymphedema worry and impact on work life and their content validity was demonstrated. Results Qualitative data from in-depth and cognitive interviews with 15 (age 40–74 years) and 16 (age 38–74 years) women with BRCL, respectively, and feedback from 12 clinical experts, were used to develop and demonstrate the content validity of six LYMPH-Q UE scales measuring symptoms, function, appearance, psychological, information, and arm sleeve. Additionally, data from in-depth interviews with 12 (age 35–72 years) women with UE lymphedema and four focus groups (n = 16 women; age 35–74 years) was used to develop and assess the content validity of two new LYMPH-Q UE scales measuring lymphedema worry and impact on work life. The content validity of the previously established six scales was also demonstrated in these subsequent qualitative studies. Conclusion The LYMPH-Q UE is a modular PROM developed using international guidelines for PROM development and can be used in clinical practice, research, and quality improvement to enhance patient-centered shared decision-making. This study’s innovative and iterative approach to content validation demonstrates that the LYMPH-Q UE is a comprehensive measure that includes important concepts relevant to patients with UE lymphedema. Supplementary Information The online version contains supplementary material available at 10.1186/s41687-024-00701-3.


Introduction
Breast cancer-related lymphedema (BCRL), which results from oncologic treatment-related disruption to the lymphatic system, is one of the most common causes of upper extremity lymphedema in developed nations.A recent meta-analysis estimated that one in five breast cancer survivors will develop BCRL, and the risk of developing lymphedema increases for up to 2 years after the cancer diagnosis or surgery [1].BCRL manifests as upper extremity swelling, heaviness, pain, tightness, skin changes, and reduced arm mobility.These symptoms and function-related impairments are often progressive and associated with a range of physical, emotional, and social sequelae impacting women's overall health-related quality of life (HRQL).The management of BCRL requires a multidisciplinary approach and may consist of non-surgical (e.g., compression, manual lymphatic drainage, exercise) and surgical (e.g., lymph node transplant, lymphovenous bypass, liposuction) interventions [2].Accurate and timely assessment of the presence and severity of lymphedema is critical to preventing the worsening of BCRL.
BCRL is assessed using patient history, review of risk factors, clinical examination including observation, palpation of the arm for pitting edema, stemmer sign, arm circumference, diagnostic tests such as lymphoscintigraphy, Indocyanine green (IGC)-enhanced near-infrared fluorescence, and patient-reported outcomes [2][3][4].While clinical examination and diagnostic tests provide valuable information, they do not capture the multidimensional HRQL impact of BCRL.Patient-reported outcome measures (PROMs) are instruments designed to capture the range of HRQL concepts that can be best known by asking patients without any interpretation by a clinician or anyone else.Recent systematic reviews [5,6] have identified several PROMs explicitly developed for upper extremity lymphedema, including the Lymphedema Quality of Life Questionnaire (LYMQOL) [7], the Upper Limb Lymphedema 27 Questionnaire (ULL-27) [8], the Lymphedema Functioning, Disability and Health Questionnaire (Lymph-ICF) [9,10], and the Lymphedema Symptom Intensity and Distress Survey -Arm (LSIDS-A) [11].A common limitation that underpins most of these PROMs is that they were developed with minimal patient input and did not follow established guidelines [12,13].Further, they do not capture the full range of HRQL issues that matter to women with BCRL [5,6].
To address these gaps, our team developed an upperextremity lymphedema-specific PROM called the LYMPH-Q Upper Extremity (LYMPH-Q UE) which has been previously published [14].This modular, conceptdriven PROM was developed using extensive patient input, followed best practice guidelines for PROM development [12,13,15,16], and utilized a modern psychometric approach (i.e., Rasch Measurement Theory (RMT) analysis) [17,18].This paper describes the multistep iterative qualitative approach to developing the LYMPH-Q UE conceptual framework (see Fig. 1) and set of independently functioning scales.

Methods
The ethics approval for the study was obtained from the Hamilton Integrated Research Ethics Board (Juranvinski Cancer Center (JCC), Hamilton, Ontario, Canada) and from the human research ethics boards of Toronto General Hospital (TGH; Toronto, Ontario, Canada), Memorial Sloan Kettering Cancer Center (MSK; New York, New York, U.S.) and Brigham and Women's Hospital (BWH; Boston, Massachusetts, U.S.).In Denmark (DK), the study was reported to and approved by the Region of Southern Denmark and was included on the list of Health Research for data protection safety.Written and verbal consent was obtained from all participants before the interviews.Study participants in Canada and the U.S. were given a $50 (CAD, USD) gift card to thank them for participating.

Approach
We used the health services research-specific qualitative approach called interpretive description [19] for this study.The development and content validation of the LYMPH-Q UE module occurred in a multi-step and iterative manner (Fig. 1).Each of these steps is described below.

Step 1: Concept Elicitation 1-for initial LYMPH-Q UE scales
A series of in-depth concept elicitation interviews were conducted between January 2017 and June 2018 with a maximum variation sample of English-speaking, adult (18 years or older) women with breast cancer who varied by age, stage, and treatment of breast cancer.The primary objective of these interviews was to create a Utility module for the BREAST-Q, and the detailed protocol for the BREAST-Q Utility module development study is published elsewhere [20,21].The interviewer contacted the participant to explain the study procedures, obtained consent and conducted the interview (by telephone or in person).During the interviews, in-depth information on the impact of diagnosis and treatment of breast cancer on participants' HRQL was elicited-(see Supplementary Material Files, Appendix 1 for interview guide).Interviews continued until data saturation was reached.Data were coded in Microsoft Office Word using a lineby-line approach and transferred to Excel using Doctools© for further refinement using constant comparison.An item pool was developed from the codes for use in scale development.
The data analysis led to the development of the BREAST-Q Utility module and identified gaps in the BREAST-Q content.One of the gaps was the limited coverage of concepts relevant to arm lymphedema.The data from the subset of participants with BCRL in this sample who provided rich information about their BCRLrelated experiences were used to draft the LYMPH-Q UE.The LYMPH-Q UE (version 1) consisted of five upper extremity lymphedema-specific scales that measured symptoms, function, appearance, life impact, and information.For each scale, the instructions, a time frame for answering, and a set of response options were drafted.

Step 2: Pilot Testing 1-for initial LYMPHQ UE scales
A series of cognitive debriefing interviews were conducted with English-speaking women with BCRL from JCC, MSK, and DK to refine and establish the content validity of five LYMPH-Q UE scales.The "think aloud" technique [22] was used, and patients were asked to comment on the comprehensibility of each component of the scale (i.e., instructions, timeframe, response options, and items) and the comprehensiveness and relevance of the items and the scale [15,16].At the end of each scale, participants were asked to describe any concepts they thought were missing.
The cognitive debriefing interviews took place in three rounds, with changes made to the LYMPH-Q UE Fig. 1 Conceptual framework of the LYMPH-Q Upper Extremity Module between rounds.Interviews in Rounds 1 and 3 were in English.These interviews were recorded, transcribed, and analyzed line-by-line.Interviews in Round 2 were in Danish [23] and were not recorded due to the need for translation before coding.Instead, for these interviews, the qualitative interviewer made detailed notes, which were reviewed by the study team and used to make revisions.Feedback was sought from a group of BCRL experts known to the investigators after Round 2. A research team member sent an email invitation with a copy of the LYMPH-Q UE scales.Experts were asked to provide written feedback via email and to add missing concepts.One reminder email was sent after 1-week.Patient and expert input was used to refine the LYMPH-Q UE and demonstrate content validity.

Step 3: Concept Elicitation 2-for lymphedema worry and impact on work concepts
The expert consults identified the need for two additional scales measuring lymphedema worry and impact on work.As the Step 1 concept elicitation was not targeted to BCRL, we did not have saturation for these two concepts.Consequently, a new series of qualitative interviews were conducted with English-speaking women with BCRL recruited from JCC between July and December 2020 to probe these concepts.The interview guide for this study is included in the Supplementary Material Files (see Appendix 2).The recruitment followed the procedures described in Step 1.All the interviews were conducted over the phone due to the COVID-19 pandemic.Interviews were audio-recorded, transcribed, and coded using the approach described in Step 1.

Step 4: Focus Groups-content validity for all HRQL concepts
As part of a separate study to understand patient priorities and preferences for upper extremity lymphedema research [24], focus group interviews were conducted over secure, encrypted and institutionally approved Zoom video-conferencing platform between May 2021 and November 2021, with English-speaking women with BCRL recruited from TGH and JCC.These interviews included women with UE lymphedema who were managed conservatively or surgically or had had surgery for lymphedema.The recruitment followed the procedures described in Step 1.A section of the focus group sessions had participants describe the impact of UE lymphedema on their HRQL regarding physical symptoms, social life, work, appearance, emotional distress, and sexual wellbeing (Supplementary Material Files, Appendix 3 for the interview guide).Focus group sessions were audiorecorded, transcribed, and analyzed using the line-byline approach described in Step 1.The HRQL data from the focus group sessions was mapped to the content of the 6 LYMPH-Q UE scales to provide evidence of content validity and support the development of two new scales for lymphedema, lymphedema worry and impact on work.

Step 5: Pilot Testing 2-for two new LYMPH-Q UE scales (lymphedema worry and impact on work)
The methodology described in Step 2 (Pilot testing 1) was followed.Cognitive debriefing interviews were conducted in two rounds with English-speaking women with BCRL (managed with or without surgery) recruited from the focus group (Step 4) sample between February and March 2022.Expert feedback was sought between the rounds using the methods described in Step 2. Patient and expert feedback was used to refine the scales and establish content validity.

Rigor
The interviews were transcribed by a professional, thirdparty company for all the steps.The data collection and analysis occurred concurrently such that new concepts were added to the interview guide iteratively.All interviews were independently coded by two coders (Steps 1 and 3) or coded by one coder and checked by another (Steps 2 and 4).The coders regularly met to review the codebook and reach consensus on coding discrepancies.The codes and the evolving conceptual framework were reviewed in research team meetings.

Step 1: Concept Elicitation 1-for initial LYMPH-Q UE scales
Qualitative interviews were performed with 57 patients in the larger BREAST-Q-Utility module study.Data from 15 participants with confirmed or suspected BCRL (i.e., patients in whom chronicity of arm lymphedema has not been established or in whom arm swelling or other symptoms of BCRL are being monitored) was used to develop the LYMPH-Q UE scales.These participants were aged between 40 and 74 years, mainly White (n = 13) and married (n = 10).Most had a mastectomy (n = 10) and a history of having combination treatment with chemotherapy, radiotherapy, or endocrine therapy (n = 7).Table 1 shows the sample characteristics.Analysis of the qualitative data for this subset of participants led to the development of a conceptual framework that included top-level domains with two or more of the following major themes: arm appearance (body image, characteristics, clothing), physical (function, symptoms), psychological (distress, impact), social (support, function, relationships) and experience of care (information) and treatment (sleeve) (Fig. 2).
The item pool was used to develop five preliminary scales for the LYMPH-Q UE Module with 57 items: symptoms (n = 18), function (n = 7), appearance (n = 11), life impact (n = 9), and information (n = 12).Each scale was assigned instructions, a time frame for responding, and four response options that measured severity (symptoms, life impact), bother (appearance), difficulty (function), or satisfaction (information).Table 1 in Appendix 4 (Supplementary Material Files) shows illustrative quotes from the patients for these concepts.

Step 2: Pilot Testing 1-for initial LYMPH-Q UE scales
Sixteen women with BCRL took part in a cognitive debriefing interview; Round 1 included two U.S. participants, Round 2 included 10 Danish participants, and Round 3 included one Canadian and three U.S. participants.The participants were aged between 38 and 74 years, mainly White (n = 16) and married (n = 11).Most participants had a mastectomy (n = 10), ALND (n = 14), and a history of having a combination of chemotherapy, radiotherapy, and endocrine therapy (n = 11).Table 1 shows the sample characteristics.Feedback was obtained from 12 of 22 (response rate: 55%) invited multidisciplinary experts.Experts came from four countries (Canada, Denmark, Poland, and the United Kingdom) and included eight plastic surgeons, two breast surgeons, a medical oncologist, and a nurse practitioner.Table 2 provides a summary of scale item revisions during each round.Overall, the scales' content was deemed easy for participants to understand.Participants only specifically asked for clarification for two items, both of which were dropped.The instructions were generally easy to understand.To the Appearance scale, after Round 3, we added instructions to make sure that women who wear an arm sleeve know to answer thinking of how their arm looks without the arm sleeve.
After Round 1, two new scales were added to measure satisfaction with arm sleeve and psychological function (see Table 2 in Supplementary Material Files, Appendix 4 for patient quotes).Both concepts were identified as important concerns during the first round of cognitive interviews and considered a gap by the research team.Data from the initial qualitative and cognitive interviews were used to create content for the scale.A summary of changes is provided in Table 2.The field-test version included 110 items: symptoms (n = 20), function (n = 19), appearance (n = 14), life impact (n = 11), psychological (n = 19), information (n = 13), and arm sleeve (n = 14).This version of the LYMPH-Q UE was translated into Danish [23] following best   practice guidelines [25,26], and the content validity of the scales was established.The psychometric findings for the new scales are published elsewhere [14].Briefly, data were collected from 3222 patients (n = 2858, Denmark; n = 364, U.S.) as part of an international field-test study.One scale (life impact) was dropped due to poor psychometric performance.The final six scales measured symptoms, function, appearance, psychological function, and satisfaction with information and with arm sleeves.Table 4 shows the characteristics of the six LYMPH-Q UE scales, including the number of items, response options, recall period, and Flesch-Kincaid grade reading level.

Step 3: Concept Elicitation 2-lymphedema worry and impact on work concepts
A total of 12 interviews were completed.The participants were aged between 35 and 72 years.Table 1 shows the demographic and clinical characteristics of this sample.In addition to the concepts of interest (see Table 3 in Supplementary Material Files, Appendix 4 for illustrative patient quotes), participants elaborated on other HRQL issues that mattered to them.The interview data supported the content of the six LYMPH-Q UE scales developed in Step 1 and 2.

Step 4: Focus Groups-content validity for all HRQL concepts
Four focus group sessions were held with a total of 16 participants (BCRL, n = 14) with UE lymphedema; the number of women who took part in each focus group was six (Session 1), four (Session 2), four (Session 3), and three (Session 4).Two participants also had leg lymphedema (one each in Session 1 and 4).For these participants, information about their leg lymphedema was not coded.For two participants, their UE lymphedema was related to ovarian cancer and melanoma treatment.The focus group sample was aged between 35 and 74 years.Twelve patients had a complete axillary lymph node dissection, two had sentinel lymph node biopsy, and two others were unsure of the type of lymph node surgery they had received.One participant had lymphedema in both arms.All participants wore a compression sleeve/bandage on their arm, and most had manual lymphatic drainage and did exercise prescribed by a physiotherapist or other healthcare professional.Table 1 shows the sample characteristics of the focus group participants.Codes on the impact of lymphedema on HRQL (e.g., physical symptoms, social life, work, appearance, emotional distress, and sexual well-being) were used in the new scale development and to add further evidence of content validity for existing scales (See Tables 1-3 in Supplementary Material Files, Appendix 4 for illustrative patient quotes).

Step 5: Pilot Testing 2-for new lymphedema worry and impact on work scales
Cognitive debriefing interviews were conducted with seven patients with BCRL from the focus group cohort (January-March 2022) to assess the two new scales' relevance, comprehensiveness, and comprehensibility.Five interviews were conducted in Round 1, one interview in Round 3, and one interview in Round 4. Twelve clinical experts also reviewed the scales and provided feedback on item relevance and comprehensiveness (Round 2).Table 3 provides a summary showing multiple revisions to the scale instructions, response options, and items in response to the feedback received by patients and experts.A total of 42 items were reviewed in Round 1.Of these, 26 were retained, 12 were revised, four were dropped, and one question was added.In Round 2, three items were dropped, one was added, and all the remaining items were revised to change the verb tense.In Round 3, two additional items were added and one revised, and in Round 4, one item was dropped while the rest were retained.The final field test version of the scales includes 17 items in the impact on work scale and 21 in the worry scale.The response options were modified from agreement to frequency (never, rarely, sometimes, often, always) based on feedback in Round 2. The recall period of "in the past week" was included for the lymphedema worry scale.The wording of the scale instructions was revised accordingly.Table 4 shows a summary of all LYMPH-Q UE scales.

Discussion
PROMs are increasingly used in clinical research and practice.When choosing a PROM, high content validity is vital to measuring change following an intervention.The in-depth qualitative interviews with patients with upper extremity lymphedema and the modular approach used to develop the LYMPH-Q UE allowed for a systematic and iterative process of developing and refining scales.It enabled us to generate additional qualitative evidence to demonstrate the content validity of the LYMPH-Q UE scales developed in Steps 1 and 2, consequently ensuring that the scales remain "fit for purpose" in different subsets of patient participants.The modular approach also facilitated flexibility to developing and validating new scales to fill conceptual gaps in measurement as they were identified.This iterative approach to the development of a PROM and the demonstration of content validity is seldomly documented in the health services research literature, although common in education measurement.Content validity is the most important measurement property of a PROM, as without it, other measurement properties such as reliability, validity, and responsiveness are meaningless.However, evaluation of content validity should not be a one-time process.It is typically examined during PROM development and pilot testing; however, this research and our prior work [27] show that content validity should be periodically reviewed, especially if new treatments become available or clinical knowledge evolves, causing changes in the content domain.Furthermore, as was the case with LYMPH-Q UE, feedback from patients and LYMPH-Q users identified gaps in the measurement of lymphedema worry and impact of lymphedema on work life, leading to the development of two new scales.Hence, routinely assessing the PROM's alignment with the content domain helps maintain the quality and relevance of measurement.
The readability of the LYMPH-Q UE was assessed using the established Flesch-Kincaid (FK) grade level.The FK grade level indicates the comprehension difficulty of written text and provides a numerical score corresponding to the U.S. school grade level [28].The FK grade level has been criticized for its focus on sentence length and syllable count, as well as its lack of accounting for the structural and semantic complexity of sentences.Further, similar to other commonly used readability scores, such as the Simple Measure of Gobbledygook (SMOG) readability formula and the Coleman-Liau Index, the FK grade level has been criticized for oversimplifying the complexity of reading comprehension [28].Nonetheless, the FK grade level is a commonly used measure and can be generated in Microsoft Word (i.e., without complex programs or software).It is recommended that more than one readability score be used to evaluate the reading grade level of written text; however, a comprehensive readability analysis of the LYMPH-Q UE is beyond the scope of this paper.
Our study had some limitations.The initial qualitative sample involved women from only the U.S. and Canada.The cognitive debriefing interviews included Danish women; however, the interviews in Danish were not audio-recorded for pragmatic reasons, as translation is time-consuming and expensive.While the LYMPH-Q UE's content validity was demonstrated in U.S., Canada, and Denmark, it is recommended that the content validity should be re-evaluated when the LYMPH-Q UE is used in a different context (e.g., country, language) and different population (i.e., non-BCRL) [12,13].Another limitation was the lack of any clinical measure of the severity of arm lymphedema for participants in the qualitative interviews, cognitive debriefing interviews, and focus groups.However, our study included women with self-reported mild to severe lymphedema and women for whom BCRLwas managed conservatively or surgically.

Conclusion
The six scales of LYMPH-Q UE module were field tested and are free for not-for-profit clinical research, clinical care, and quality improvement initiatives through http:// www.qportfolio.org.The new LYMPH-Q UE lymphedema worry and impact on work-life scales are currently being field-tested.This study's innovative and iterative approach to content validation demonstrates that the LYMPH-Q UE is a comprehensive measure that includes important concepts relevant to patients with UE lymphedema.

Fig. 2
Fig. 2 Iterative concept elicitation and content validation of the LYMPH-Q Upper Extremity Scales

Table 1
Characteristics of participants Data for one U.S.A cognitive debriefing interview participant is missing a Focus groups included one participant with melanoma and one participant with ovarian cancer

Table 2
Step 2-Pilot Testing 1-summary table showing changes to each scale

Table 3
Step 5-Pilot Testing 2-summary table showing changes to impact on work and lymphedema worry scales

Table 4
Description of LYMPH-Q | Upper Extremity Scales N/A Not Applicable